Applying RE-AIM to evaluations of Veterans Health Administration Enterprise-Wide Initiatives: lessons learned.

Introduction: The United States Veterans Health Administration (VHA) Office of Rural Health funds Enterprise-Wide Initiatives (system-wide initiatives) to spread promising practices to rural Veterans. The Office requires that evaluations of Enterprise-Wide Initiatives use the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. This presents a unique opportunity to understand the experience of using RE-AIM across a series of evaluations. The authors conducted a study to document the benefits and pitfalls of using RE-AIM, capture the variety of ways that the team captured the elements of RE-AIM, and develop recommendations for the future use of RE-AIM in evaluation. Materials and methods: The authors first conducted a document review to capture pre-existing information about how RE-AIM was used. They subsequently facilitated two focus groups to gather more detailed information from team members who had used RE-AIM. Finally, they used member-checking throughout the writing process to ensure accurate data representation and interpretation and to gather additional feedback. Results: Four themes emerged from the document review, focus groups, and member checking. RE-AIM: provides parameters and controls the evaluation scope, "buckets" are logical, plays well with other frameworks, and can foster collaboration or silo within a team. Challenges and attributes for each RE-AIM dimension were also described. Discussion: Overall, participants reported both strengths and challenges to using RE-AIM as an evaluation framework. The overarching theme around the challenges with RE-AIM dimensions was the importance of context. Many of these benefits and challenges of using RE-AIM may not be unique to RE-AIM and would likely occur when using any prescribed framework. The participants reported on the RE-AIM domains in a variety of ways in their evaluation reports and were not always able capture data as originally planned. Recommendations included: start with an evaluation framework (or frameworks) and revisit it throughout the evaluation, consider applying RE-AIM PRISM (Practical Robust Implementation Framework) to gain a broader perspective, and intentionally integrate quantitative and qualitative team members, regardless of the framework used.

Common Ground: Primary Care and Specialty Clinicians' Perceptions of E-Consults in the Veterans Health Administration.

OBJECTIVE: The US Department of Veterans Affairs (VA) introduced electronic consultation (e-consult) to increase access to specialty care. The objective of this study was to understand perceptions of e-consults that may be relevant to increasing adoption in the VA. METHODS: Deductive and inductive content analysis of semistructured qualitative telephone interviews with VA primary care practitioners (PCPs), specialists, and specialty division chiefs was performed. Participants were identified based on rates of e-consult in 2016 at the individual and facility level within primary care, hematology, cardiology, gastroenterology, and endocrinology. Interview guide development was informed by the Practical, Robust, Implementation, and Sustainability (PRISM) framework. RESULTS: We interviewed 35 PCPs and 25 specialists working in 36 facilities. Four themes emerged across both PCPs and specialists: (1) e-consults are best suited for certain types of clinical questions; (2) high-quality e-consults include complete background information from the requesting clinician and clear diagnostic or treatment recommendations from the responding clinician; (3) PCPs and specialists perceive e-consults as a novel opportunity to provide efficient, transparent care; and (4) lack of awareness of e-consults hinders adoption despite obvious benefits. CONCLUSIONS: We identified themes that are informative for further adoption of high-quality e-consults in the VA. Educating PCPs and specialty practitioners about the benefits of e-consults, and providing support, such as lists of specialties available for e-consults at the facility are 2 such practices.

Adaptations to relational facilitation for two national care coordination programs during COVID-19.

Background: Adaptations to implementation strategies are often necessary to support adoption and scale-up of evidence-based practices. Tracking adaptations to implementation strategies is critical for understanding any impacts on outcomes. However, these adaptations are infrequently collected. In this article we present a case study of how we used a new method during COVID-19 to systematically track and report adaptations to relational facilitation, a novel implementation strategy grounded in relational coordination theory. Relational facilitation aims to assess and improve communication and relationships in teams and is being implemented to support adoption of two Quadruple Aim Quality Enhancement Research Initiative (QA QUERI) initiatives: Care Coordination and Integrated Case Management (CC&ICM) and the Transitions Nurse Program for Home Health Care (TNP-HHC) in the Veterans Health Administration (VA). Methods: During 2021-2022, relational facilitation training, activities and support were designed as in-person and/or virtual sessions. These included a site group coaching session to create a social network map of care coordination roles and assessment of baseline relationships and communication between roles. Following this we administered the Relational Coordination Survey to assess the relational coordination strength within and between roles. COVID-19 caused challenges implementing relational facilitation, warranting adaptations. We tracked relational facilitation adaptations using a logic model, REDCap tracking tool based on the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) with expanded Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) dimensions, and member checking. Adaptations were analyzed descriptively and for themes using matrix content analysis. Results: COVID-19's impact within the VA caused barriers for implementing relational facilitation, warranting eight unique adaptations to the implementation strategy. Most adaptations pertained to changing the format of relational facilitation activities (n = 6; 75%), were based on external factors (n = 8; 100%), were planned (n = 8; 100%) and initiated by the QA QUERI implementation team (n = 8; 100%). Most adaptations impacted adoption (n = 6; 75%) and some impacted implementation (n = 2; 25%) of the CC&ICM and TNP-HHC interventions. Discussion: Systematically tracking and discussing adaptations to relational facilitation during the COVID-19 pandemic enhanced engagement and adoption of two VA care coordination interventions. The impact of these rapid, early course adaptations will be followed in subsequent years of CC&ICM and TNP-HHC implementation.

Impact of Longitudinal Virtual Primary Care on Diabetes Quality of Care.

BACKGROUND: Lack of healthcare access to due to physician shortages is a significant driver of telemedicine expansion in rural areas. Telemedicine is effective for management of chronic conditions such as diabetes but its effectiveness in primary care settings is unknown. OBJECTIVE: To evaluate differences in diabetes care before and after implementation of a longitudinal virtual primary care program. DESIGN: Propensity score-matched cohort study utilizing difference-in-differences analysis. PARTICIPANTS: Patients with diabetes who received care at VA primary care clinics between January 2018 and December 2019 where the Virtual Integrated Multisite Patient Aligned Care Teams (V-IMPACT) program was implemented. EXPOSURE: Patient participation in at least one V-IMPACT visit while usual care patients did not participate in V-IMPACT. MAIN MEASURES: The primary outcome was change in hemoglobin A1C (HbA1C) and secondary outcomes included change in the proportion of patients meeting diabetes quality indicators: blood pressure control, statin use, angiotensin-converting enzyme inhibitors or angiotensin II receptor blockers (ACEi/ARB) use, and annual microalbuminuria testing. KEY RESULTS: Our propensity-matched cohort included 9010 patients split evenly between those who participated in V-IMPACT and those who remained in usual in-person care. Among individuals with diabetes who participated in V-IMPACT, the change in mean HbA1C was - 0.055% (95% CI - 0.088 to - 0.022%) while those in usual care had a - 0.047% (95% CI - 0.080 to - 0.014%) change before and after program implementation. We observed a 5.1% (95% CI 2.4 to 7.7%) absolute increase in the proportion prescribed statins in the V-IMPACT group, a 5.3% (95% CI 2.5 to 8.2%) increase prescribed ACE/ARBs, and a 4.6% (95% 1.7 to 7.5%) increase in completed yearly microalbuminuria testing. V-IMPACT was not associated with a significant difference in the proportion with controlled blood pressure at < 140/90 or < 130/90 mmHg thresholds. CONCLUSIONS: Quality of diabetes care delivered by a longitudinal virtual primary care model was similar if not better than traditional in-person care.

Cost savings associated with electronic specialty consultations.

OBJECTIVES: Electronic consultations, or e-consults, between primary care providers and specialists have been shown to improve access to specialty care, shorten wait times, and reduce outpatient visits. The objective of this study was to evaluate differences in health care costs between patients who received an electronic specialty consultation and patients who received a face-to-face specialty consultation. STUDY DESIGN: Retrospective cohort evaluation of patients who received a specialty consultation in the Veterans Health Administration during 2016. METHODS: Patients who received an e-consult were matched 1:1 to patients who received a face-to-face consultation using propensity scores. Total, outpatient, and inpatient health care costs over 3 and 6 months following the specialty consultation were compared using a generalized linear model with a gamma distribution and log link. RESULTS: e-Consults accounted for 1.8% (urology) to 9.6% (hematology) of specialty consultations, on average. Across 11 specialties, patients receiving an e-consult had significantly lower health care costs compared with patients receiving a face-to-face consultation, ranging from 3.6% (cardiology) to 30.7% (hematology) lower. This was largely driven by differences in outpatient costs. Patients receiving an e-consult had significantly lower outpatient costs for all specialties except cardiology, ranging from 6.9% (endocrinology) to 31.2% (hematology) lower. Three-month inpatient costs among those who received an e-consult were significantly lower only in cardiology (5.2%), nephrology (9.3%), pulmonary (13.0%), and gastroenterology (14.3%). CONCLUSIONS: Electronic specialty consultations are a potential mechanism to reduce health care costs and promote the efficient use of health care resources.

Online Versus Face-To-Face Training of Critical Time Intervention: A Matching Cluster Randomized Trial.

This study examined the effectiveness of online education to providers who serve people experiencing homelessness, comparing online and face-to-face training of Critical Time Intervention (CTI), an evidence-based case management model. The authors recruited 184 staff from 19 homeless service agencies to participate in one of two training conditions: (a) Online Training + Community of Practice or (b) Face-to-Face Training + Telephone Consultation. Each group received 24 hours of instruction and support. Through baseline, follow-up, and nine-month post-training surveys, the authors examined satisfaction, knowledge gains, knowledge retention, and readiness to implement CTI. While satisfaction rates were higher among participants in the face-to-face group, the two training conditions produced comparable pre/post knowledge gains. Furthermore, both groups showed increased knowledge retention scores at nine-month follow up, with the online group scoring higher than the face-to-face group.